Estimation and Control of the False Discovery Rate of Bayesian Network Skeleton Identification

An important problem in learning Bayesian networks is assessing confidence on the learnt structure. Prior work in constraint-based algorithms focuses on estimating or controlling the False Discovery Rate (FDR) when identifying the skeleton (set of edges without regard of direction) of a network. We present a unified approach to estimation and control of the FDR of Bayesian network skeleton identification and experimentally evaluate the performance of a standard FDR estimator in both tasks over several benchmark networks and sample sizes. We demonstrate that conservative estimation and strong control of FDR are not achieved in some cases due to insufficient sample size and/or unfaithfulness. We show that a permutation-based and a parametric-bootstrapbased FDR estimator achieve more accurate FDR estimation and strong control than the standard estimator. Finally, we present a relaxed definition of false positive that leads to more conservative estimation and control of FDR in relatively small sample sizes.